Systems and methods for the automatic detection, analysis and recognition of facial features find application in a wide range of fields, including, for example, automatic facial and silhouette recognition, identification and tracking systems employed in security system applications. Such systems may include, for example, identification badge systems, personnel recognition systems and area security systems, including systems operating in densely populated areas such as airports and other transportation and shipping facilities and entertainment and sports venues. Such methods and systems also find a wide range of applications in such commercial and consumer applications as image and photograph processing systems, including automatic color correction systems, photographic editing systems and automatic photomontage systems.
Facial feature recognition systems may employ a variety of methods to identify facial images appearing in a scene and to identify the component elements of each face, including such methods as morphological analysis, pattern recognition and image template matching methods. One of the most frequently used methods, however, is hair segmentation, which is a process by which that portion or portions of an image that represents hair can be distinguished from other portions of an image that may represent, for example, skin, clothing, or other objects.
Conventional hair segmentation methods of the prior art, however, have primarily been designed to distinguished between hair and skin by detecting differences between the visual characteristics of skin and hair to thereby define the boundary of facial skin areas in an image. The methods of the prior art, however, rely upon there being a strong value contrast or other strong visual characteristic difference, such as color, between the facial skin and hair or between hair and, for example, background elements of the image.
For example, the method described in U.S. Pat. No. 5,631,975 to Riglet et al. for an “Image Segmentation Device” describes a method for identifying areas of an image containing hair by using luminance differences to identify the skin/hair and hair/background boundaries. U.S. patent application Ser. No. 20050117779(A1) by Daisaku Horie and Yuusuke Nakano for an “Object Detection Apparatus, Object Detection Method and Computer Program Product” describes a method using chromaticity values to detect flesh and hair areas of pedestrian images. U.S. patent application Ser. No. 20050175227(A1) by Stavros Paschalakis for a “Method and Apparatus for Separating Hair and Skin in Images” describes a method for identifying hair areas of an image by identifying and excluding skin colored pixels from a skin map. In a further example, U.S. Pat. No. 5,430,809 to Tomitaka for a “Human Face Tracking System” and U.S. Pat. No. 5,812,193 to Tomitaka et al. for a “Video Camera System Which Automatically Follows Subject Changes”, identify areas of skin and hair in an image by tesselating an image into tiles, comparing the colors, hues or hue angles of each tile area with standard skin hues, and identifying as hair areas those tiles having skin hues of sufficient darkness.
More recently developed methods of the prior art for hair segmentation in images employs the above described methods based on skin/hair/background luminance, hue and chromaticity values with methods for recognizing and identifying the geometric structure of a facial image. For example, U.S. Pat. No. 6,862,374 to Yoshinori Nagai, Hajime Takezawa, Kazuhiro Saiki, Toshiya Takahashi and Kyouichi Suzuki for an “Image Processing Device, Image Processing Method, and Recording Medium Storing the Image Processing Method”, describes a method wherein an image of a face is represented as a simple geometric shape against a uniform background and image pixels are identified as containing or not containing hair by pixel chromaticity, and hair regions are identified by hair/no hair border tracing within smaller windows.
“Detection, Analysis, and Matching of Hair”, Yasser Yacoob and Larry Davis, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 pp. 741-748 is primarily focused on methods for the calculation of hair similarity after the hair has been segmented, but includes some description of a hair segmentation method. According to Yasser Yacoob and Larry Davis, the hair segmentation method is comprised of the steps of (a) detecting within an image a frontal image of a face against a plain background, (b) detecting the location of eyes within the face shape, (c) modeling the skin color within the face shape by color analysis, and (d) modeling the hair color by color analysis. It should be noted, however, that Yasser Yacoob and Larry Davis state that the method has difficulty in differentiating the hair/background border and with subjects having little or no hair.
In a final example, “A Model-based Approach to Hair Region Segmentation”, Zhi-Qiang Liu and Jessica Y. Guo, International Journal of Image and Graphics, Volume 3, Number 3 (2003), pages 481-501, describes a method for hair segmentation that relies upon a frontal facial image and a strong contrast between the outside border or the hair and the image background and describes morphological processes that may be used to generate a hair region map.
The methods of the prior art thereby rely upon there being a substantial contrast or other strong visual characteristic difference, such as color, between the facial skin and hair or between hair and, for example, background elements of the image. As a consequence, these methods of the prior art often ineffective or unreliable in situations wherein the contrast or visual characteristics differences between the areas of image are not strong or where some areas of an image, such as the background, are complicated or cluttered. It should also be noted that many if not most of the face recognition methods of the prior art also require that the facial images be substantially frontal views of the subject's face in order to identify and segment the various areas of the subject's face.
Therefore, while such methods of the prior art have shown some limited capability in distinguishing areas of skin from areas of hair, they have also shown a limited capability in distinguishing hair from other areas appearing in an image, such as background elements, and particularly in cluttered images such as found in consumer/commercial photography and in securing systems dealing with densely populated areas. Consumer images and images of densely populated areas, however, typically include multiple people in any number of positions or poses and against any of a number of types of backgrounds and backgrounds having considerably varied content, including backgrounds containing large numbers of closely spaced or even overlapping faces. Moreover, hair colors and styles can vary widely across the population and methods that use only luminance, chromaticity, hue, or other color space data can be inadequate to the task of hair segmentation for images obtained by the broad base of digital camera users. It should also be noted that not only do the methods of the prior art require uniform backgrounds and impose tightly constrained facial positions and angles, they also show little or no capability in dealing with factors such as subject age or gender and the correlation of these factors to the probable distribution of hair belonging to the subject. The systems of the prior art, for example, often exhibit difficulty in image assessment with subjects having little or no hair or facial hair such as a beard or moustache.
The present invention provides a solution to these and other related problems of the prior art.